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I* Introduction and J ^erriev 

This report siinmi£ri:ze3 wcr:; dcrs -^rr-r a period of 14 monizhs reviewing 
early Initiatives by 'ZEJA .^x±v^-^ .nii^ar^z—- in knowledge Development under 
the Youth Empioyusfint air Demonsr? i;z.rr. : precis Act of 1977. It is based 
primarily on two waves o-; site ^v^;:it: .1 ^mtal of 19 prime ^ponsors^ ~ the 
first wave in the month:^ 2f Ms-^^--: me J- and the second wa^e in the 
months of Noveraber-Jariuary, l^^r-iQ. 

In the spring of ir ?, pr; c:. zz .e site visits, MDC central staff 
had looked in on the 10 Depart??^: of l^bor regional offices. Our 
objectives were, first, no gei for regional staff attitudes 

toward Knowledge Develcxnrent , amd seccr r to get a list of those prime 
sponsors most likely tc ave a :ivs Knowledge Development initiatives. 
We also established cd^ct wi - Osorc; and Associates, a private firm 
which had contracted ts:::^:: OY? ' conduct: a series of conferences to 
provide local prime sr£>n:ri6rs v±rh training in Knowledge Development 
techn?.ques. From our ^ . .ts vrith the rev:ional offices and Osoro, we 
compiled a roster of rnin£::rions . 

Based on this iirSrr: cioz^. MDC engaged faculty of the School of 
Public Administration az 1. -rth Carol±aa Central University to develop 
project abstracts, via tete^hone calls, of the most likely sounding 



Richmond, California:; San Francisco, California; Escambia County, 
Florida; Montgomery. Alabama; Chicago, Illinois; Dcnnison, Texas; Corpus 
Chrlstl, Texas; San Antionio, Texas; Clark County, Washinp.ton ; St . Louis, 
Missouri; Montana B-O-S; ^ayne County, Hichip.nn; Clackamas County, Oregon; 
Yakima, Washii. j;ton; Fayetre, Pennsylvania; Dallas, Texas; Atlanta, 
Georgia; Penobscot, Malnp-. Jackson-Josephine, Orep,on. 
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Knowi^5ic5e Development initiativer , From the^: abstracts, 1: sites 
were 'rhj^sen for two-day visits b- :iDC staff rz;: field research associates. 

,s report, then, 1^ base; .argely on .:..:vr:r:2ation gained ;;y brief 
Mite T:u3lts to prime spoinTx%rshr.pr believed tc mavr- somewhat ad-snced 
Knowledge Development act- -;^^s imder way ir -or i^tion with their 
youth r-nrrograms. As will ppsrsnt from reacxi: ^ tne report, however^ 
most of the Knowledge Der:: 2nt we found was t an early stage. 
This is, of course, unde^^ta: .le in light cf t^tie early dates — fully 
a year ago — at which ' - vi^.. ...s were made. 7t regional of i ices 
were just beginning to : . a Die in spreading ~ Knowledge Develop- 
ment 1^ tiative among t .r primes, and the Osc training project 
was barely under way. weral of the regions F*ez:]ied to be developing 
a good understanding of .e KD concept; but, fr the most part, informa- 
tion from the regional _:fiic^*^.y was rather sketchy. In some cases, 
Knowlpcge Development activitie;^ that looked grrod on paper and sounded 
good crirsr the telephone eith-ar proved insuf f icieritly advanced to yield 
reSL^tx or fell through for one or another reason. 

This is perhaps not surprising in view of the fact that HDC was 
pursuing an initiative that had not been defined and waci not mandated in 
the strict sense of that word. Prime sponsors were left free to ignore 
Knowledge Development and some did. They were also free to create 
Knowledge Development plans on paper, and then never carry them out, 
and ^ome did that too. 

Generally, HDC found activity of one sort or another aimed at 
satisfying what seemed to both regional staff and state and local prime 
sponsors an extremely nebulous goal. What was Knowledge Development 
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supposed to mean. MDC observers i;ere asked frequently, and how was one 

to go about getting it? 

For internal purposes, MDC nad decided upon a rough tentative 
definition of its own— what it thoaght Knowledge Development for youch 
programs, or any other CETA prograas, ought to mean. It ought to mean, 
we thought, the collection of infrarmacion under a plan iiitended to yielii 
knowledge that may predictably be useful in improving CETA programs for 
youth. 

We were prepared to alter the definition as we went along, to bui. . 
on it, or to make it more precise, but we were more interested in how 
the prime sporsors and program operators were defining it and—if we 
could find out — why. 

In this report we have tried to depict the various kinds of 
initiatives that have sprung up under tha rubric of Knowledge Development. 
We are avare that some of the larger prime sponsors have evaluation 
systems which essentially provide to all programs— including youth 
programs— the benefits described in our definition of Knowledge Development. 
Since these systems preceded the youth act, and since most small or 
medium-sized prime sponsors, would have difficulty replicating then, we 
did not concentrate on them. 

We have tried, instead, to provide a good, close look at early 
Knowledge Development initiatives by a few prime sponsors. Our 
conclusions, however, and our recommendations for the future, are based 
on the vork of the entire project, including our talks with regional staff, 
with Osoro. and our brief contact with prime sponsor efforts that did not 
seem likely to lead to the development of any particular knowledge 
beyond the recognition that learning anything can be hard work. 



II. The Regional Perspective 

After completing visits to the ID regional DOL offices, MDC held 
a project briefing April 26-27 » 1979, for its field research associates 
and members of the North Carolina Central University staff who were 
to prepare the abstracts. Staff memfaers who had participated in the 
regional visits were asked what they had learned. 

"They mainly asked us to tell them what we thought Knowledge Develop- 
ment was," one MDC staff person reH^onded. 

This summarization of the regional visits still seems as useful 
today as it did then. MDC staff was invariably asked by the regional 
youth coordinator, a federal representative, or some other regional 
staff to define Knowledge Development. The request was framed so as to 
suggest that since MDC had a contract from the Office of Youth Programs, 
It could speak for OYP. The experience made us feel a little like 
foreign emissaries asked to explain our government's intentions. 

Without being coy, we tried to elicit definitions from our 
interrogators, as it war* part of our study design to try to learn how 
KD was being perceived, first at the regional and then at the local 
level. Wlien this approach failed — as it often did — we shared our 
private definition with them, not as "word" handed down from OYP but 
aimply in terms of our own perceptions. Usually, regional staff accepted 
the definition as reasonable, if not definitive, and we moved on to 
other bus! less. 

But this did not caean that we were furnished examples, good or not 
so good, of Knowledge Develcfment under our definition or another. In all, 
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MDC staff developed 62 nominations from the regions. This figure, 
however, does not accurately reflect the degree of understanding exhibited 
by the regional offices. 

For one thing, the level of awareness about KD activities varied 
considerably among the regions. From three of the 10 regions we received 
no nominations. The majority of the nominations received wercj made 
largely on the basis of mention by prime sponsors in grant applications 
that a Knowledge Development activity of some kind was in progress. 

In one office — Region I ~ we received full cooperation and 
perceived a quick and impressive grasp on the part of the appropriate 
officials of what the primes were doing and what might realistically 
be called Knowledge Development. Ke were received courteously in the 
other nine offices, buL the ampunt of information we were able to gather 
about prime sponsor youth activities having to do with Knowledge Develop- 
ment ranged from modest to minimal. 

In every regional office, we talked with the youth coordinator 
and in many cases with a number of the federal representatives for the 
primes that were nominated. In most regions, however, we had to go 
to the primes themselves by telephone to get a clearer idea of what 
was really going on. 

Staff at a few of the regional offices seemed to think that Knowledge 
Development was any prograi-i directed toward some "special*' target. Vc 
were given as nominations for exemplary Knowledge Development a prograr.: 
for dropouts, a program for pregnant teenagers, and a program for handi- 
capped youth. In none of these cases had any formai or informal method 
been determined for gleaning specific lessons. Knowledge Development 
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or programinatlc content of sone sort. If it is 'W" — so this reasoning 
would seem to go — then it must be Knowledge Development. 

We sensed that most regional offices felt that they had been supplied 
with little that would help them answer the question of what Knowledge 
Develcyment was supposed to be. 

This may explain why we weje not able to get the kind of information 
we had hoped to get from the regional offices. Our study plan called 
for simply turning over the nominations to the N.C. Central staff for 
winnowing down to the 25 three-page abstracts. But where there were 
few or no nominations from the regional offices, MDC staff developed this 
basic information on the phone with the prime sponsors themselves^ 

In this process, we discovered early on that what various prime 
sponsors were doing in the nam«i of Knowledge Development frequently 
bore little resemblance to what they said they would do in their plans. 
Sometimes this discovery cast light on the level of understanding of 
Knowledge Development at a particular regional office. 
A case here may be illustrative: 

In one regional office, the MDC staff member was told by the 
regional youth coordinator that no nominations had been obtained from 
the field. This youth coordinator clearly had made an effort to obtain 
self-dominations and had been unsuccessful — he showed the MDC staff 
member the memorandum he had distributed to all primes. (The memorandum, 
Itself, was interesting in that it •called for primes which had "model" 
programs to no^ainate themselves, a declaration that most primes might 
be disinclined to make). 

8 



- 7 - 

In view of the lack of self-nominations, he suggested that the 
MDC staff member visit with several of the federal representatives to 
the primes," located in that same building. He was extremely helpful in 
introducing our man to these individuals who, in turn, were as helpful 
as they could be in suggesting which prime sponsors might have "something 
going" In KD. 

One federal representative seemed especially well informed about 
prime sponsor activity, including a good bit that was not in her area 
of week-by-week contact. She suggested some likely programs, and the 
regional youth coordinator agreed. Later the coordinator nominated 
two of these prime fr'^y^irsors^ and they were duly the object of abstracts 
and site visltF 

When our liJwt. '-^QKntr^i researcher reported on the first of tb^ise in 
his abstract, he refe^rc^a to the program as "diamond in the rough" KD. 
l^en the MDC staff member assigned to visit the prime sponsor arrived. 
It became clear what that phrase meant. The prime sponsor was I'unning 
apparently good programs but had no plan to gather knowledge about them, 
no evaluative scheme of any sort. \^hat this prime sponsor needed was 
help in getting started on Knowledge Development so that he might provide 
evidence of the apparent success of his programs and learn how to make 
them work even better for the participants. 

One way or another, MDC staff gathered sufficient information to 
provide its subcontractor with leads for the telephone survey. Based 
on information collected in this way, the subcontractor winnowed down the 
nominations to provide 25 abstracts. MDC staff chose nineteen of these— all 
that seemed likely to repay further investigation — for site visits. 

ERIC 3 
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III. The Prime Sponger Perspective 

Developing KnowlpHae about Kno wJedee DevPl op n,onf 
Confusion about Knowledge n,,velopment Is one thing at the r=sicnal 
level, at a slgnlflc^t remove from program operation; it is something 
else again at the prime sponsor level, where it Ig translated into 
program action or, as was at times the case, inaction. 

It is known that some prime sponsors did little or nothing at 
all initially about Knowledge Development. While the prime sponsors 
chosen by the regions for this project were supposed to be outstanding 
examples, comparisons with other primes MDC is familiar with suggest 
that the study primes were, at best, average. If this is the case, the 
average prime sponsor made some efforts in fiscal 1978-however faltering- 
to fulfill the Knowledge Development mandate. 

Among the 19 primes visited, initial responses to the announcement 
that a Knowledge Development section would be required for their youth 
plans varied sharply. 

In Atlanta, the prime sponsor's youth planner perceived Knowledge 
Development as "nothing new." Rather, as our field research associate 
noted: "The youth planner saw KD as a new name for what they were doing 
and would have continued to do without the directive from the national 
office." 

Staff members of the prime sponsor in Clark County (Wash.), on the 
other hand, were quite vocally disturbed about the manner in which 
the KD mandate came down to them. According to that prime's youth 

10 
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planner, he had only a nK,nth to find out what Knowledge Development was 
and to design a proposal incorporating it. He also pointed out that, 
during the harried „onth, he could get no guidance, clarification, or 
technical assistance £ron> either the regional or the national office. 

In Richmond. Calif., the new initiative also seemed overwhelming. 
"We were totally dismayed." ,he youth planner recalled. "We're a small 
town-our total allocation is only $6 million-and we had no .oney for 
research. Our goal was and still is to spend as much of our money as 
possible on direct services to youth." The feeling here seemed to be 
that efforts at evaluation could- be undertaken only with a commensurate 
loss in service. Yet despite this attitude, the Richmond prime sponsor 
managed to conduct one of the better Knowledge Development ventures 
MDC eicamined. 

The wide difference in perceptions between Atlanta on the one 
hand and Richmond and Clark County on the c.her may have been as much a 
function of prime sponsor ,ize as anything else. The larger primes 
tended either to be comfortable with the idea or to shrug it off as 
"nothing new." The smaller primes tend-d to be at a loss to know 
where to begin. 

However confused they were, they could get little real guidance 
from the regional level. A couple of vignettes should point :,p this 
Bituatlon. 

The Jackson-Josephine youth planner told MDC that neither he nor 
his superiors ever viewed KD as a "mandate." Rather, they saw it as an 
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optional suggested activity-one they were free to incorporate or ignore, 

as they saw fit. The planner noted that there vas no new money to 

help foster KD, and that there were no discernible "rewards" for the 

extra effort that would be necessary to plan, operate, and track a 

KD effort. Still, he diligently checked out his perceptions with the 

youth staff at his regional office. He describas that attempt: 

"I called my Fed Rep to ask what Knowledge ' 
Development was. He said he didn't know. 
That was that. End of conversation." 

^The Montana B-O-S experience was similar to that of Jackson-Josephine. 
When the Montana state prime called the regional office to find out what 
Knowledge Development was and how to proceed, it got something less 
than a clear response from the federal representative. After some 
scratching of their heads, the prime's planners decided to put effort 
into an approach which appears" too limited statistically to have much 
value. 

This last is not to say that what aontana tried to do was useless 
programmatically, or even that some valuable information did not emerge. 
It is to say only that there was no design for learning, no systematic 
way of gathering information so as best to provide useful learnings. 

Small wonder, really, when the dearth of evaluative practices of 
the average prime sponsor is taken into consideration. As it sifted 
through the regional suggestions. MDC found a nuir.ber of primes who simply 
had not made the most basic distinction between doing something and 
learning something about what was being done. In a number of cases. 
Knowledge Development had been defined apparently as a "new program"- 
anything that had not been tried by that particular prime before. In 
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other cases I the KD concept spurred primes to attempt to serve new 
and significant segments of the target population, as though the 
knowledge they sought was to be developed in the heads of the new 
participants. 

In yet other cases, information putatively related to KD was 
gathered, but in a willy-nilly fashion, with no clear idea of how it 
would be used. A prime sponsor told an MDC field research associate; 
"I've got batches of facts and figures. I've administered attitudinal, 
occupation, psychological tests.** When asked to what purpose, he 
responded: "Hell, I don't know, but if you want to come down here, 
we'll develop any kind of knowledge you want." 

The confusion extended, then, not just to what Knowledge Development 
was, but also to the matter of whom it was intended to help. 

Too Much Mill, Too Little Grist 
Even when prime sponsors understood that the knowledge to be 
developed was for their own benefit and that it must proceed from an 
examination of evidential data, other problems arose. After looking at 
dozens of abstracts and comparing notes on telephone surveys and site 
visits to primes, MDC believes that it can break these problems down 
Into two essent^,al types: First, there were the problems associated with 
too much structure to support altogether too littl;^ information. 

It was surprising to see how many prime sponsors seized upon the 
classical outcome evaluation model, with or wi. hout comparison groups, 
^.or their Knowledge Development effort — or perhaps "exercise" would be 
the more appropriate word. MDC has concluded that many of these primes, 
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unaccustomed to perfomiini; evaluation, had simply adapted what seemed 
to them to be a standard national model to their local purposes. 

A good example of this Is the Montana B-O-S Knowledge Development 
activity which was designed around the summer youth program* The 
project was set up to measure four different combinations of services to 
determine which package was most effective in serving a population of 
school dropouts. The service combinations chosen entailed Work Experience 
by itself; Work Experience plus counseling; Work Experience with counseling 
emphasizing school return; and Work Experience combined with Career 
Awareness. All four of the service combinations were to be tested 
at each of four sites around the state, and each test site was to have 
a control group against which the progress of the four test groups 
would be compared. 

On paper, and over the telephone, this looked and sounded like 
good, useful KD activity; it was carefully planned, sensibly staged, and 
appeared to have some very useful program learnings inherent in it. 

But in actuality the project broke down: There was simply too 
much of a research-theory "mill" for too little inCoimational grist. 

Only 53 individual dropouts were, enrolled in the project statewide — 
that is, about 14 at each of the four sites. Each of the four sub-gxoups 
at each site thus consisted of only three or four youth. Moreover, th. 
control group for all four sites together was made up of only 19 individuals 

The samples, then, were simply too small to permit the drawing of any 
valid conclusions about the effectiveness of the iseveral service mixes. 
To compound matters further, the project was put into place differently 
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at each of the four demonstration sites* And finally, the data collected 
on the "control group" was marred and incompletet 

The project snded in September of 1979. By winter of 1980, 
when MDC last checked with prime sponsor staff, no report of findings 
had yet been prepared. Given the numbers above, however, no one, 
unfortunately, need await the report with great anticipation. 

We have singled out the Montana project as a clear example of the 
danger that arises in trying to learn and generalize from a narrow 
Information base. Montana, sad to report, was far from being the only 
prime to make this fundamental error. Montana's project lends itself 
to illustration not because it accomplished so little in Knowledge 
Development, but rather because, with a sounder approach, it could have 
accomplished so much. 

Relatedly, the use of control or comparison groups — an extremely 
risky technique when employed on a small scale— proved the undoing of 
other well-meaning prime sponsors. 

Corpus Christi has done some interesting work involving comparison 
groups; but basic project orientation and the spe-.ifics of project 
Implementation have all but vitiated the body of learnings that might 
have come from the experiment. 

As part of the "ten percent income test experiment" conducted in 
1979, Corpus Christi set up a project which, through a number of facets, would 
allow it to examine the question of whether CETA-eligible youth bent-.f itted 
from interaction with youth from higher income, CETA-ineli^ible backgrounds. 

Fifty-six young people were chosen for the project — one-half 
CETA-eligible youth, and one-half higher income youth. Then one-half 
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of each subgroup was assigned to one of the two area high schools — one 
the mainstream high school and the other an alternative education center. 

Project t^taff hoped to use the comparison group situation to make 
judgments about the effects of factors such as age, sex, family income 
level, and education on youth employability. But after the project had 
been run» staff admitted that they had made but little progress towards 
the desired judgments. They concluded that they had, In effect, been 
looking at the wrong factors, and that matters such as personal 
motivation, role model exposure, and individual aspirations impinged 
more directly on employability than did the factors they set out to 
examine. 

The project did produce one significant learning, even though It 
was in the nature of a side-effect: The CETA-eligible half of the group 
stayed in the program and progressed at about the same rate as the group 
of CETA-ineligible youth. Still, chances for the other Important 
learnings were lost because of over-reliance on an improperly designed 
comparison format. Again, as with the Montana summer program, thr- 
research mechanism was far more complex and full-blown than the material 
that was fed into it. 

A final note: The Corpus Christ! project, which took on the 
trapping!, of z rather scohistlcated Knowledge Development effort, was 
not really viewed by Its operators as a KD effort at all. This becanie 
clear through conversations with the agency's youth planner, who stated: 
"The underlying reason for structuring an experimental research project 
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Involving 56 youths was to pain more resources for the youth" [our emphasis]. 
We have, then, a rather interesting example of KD activity being 
undertaken for motives which had little to do with the desire to develop 
knowledge. 

In several other cases, efforts at establishing elaborate formal 
evaluation systems have simply floundered, leaving the prime sponsor 
scrambling to snatch some sort of benefit from its efforts. The 
Fayette County, Pa., prime sponsor's effort illustrates the sort of 
rescue operation that activities have turned into. The Fa\yette 

youth demonstration, :::ir2ciied at $324,000, was to consist of two tracks: 
an in-'school counseling program for potential dropouts, and an out-of-schaol 
counseling, work experience, and remedial education track for dropouts. As 
written, the project plan was couched in teruis of being a Knowledge 
Development undertaking: It was to include strong research and evaluation 
components. Learnings were to be generated about track-specific 
effectiveness through the u^'^e of control groups, pre- and post-progran 
testing, and other measurement techniques. 

The project looked good at the outset. Thirty youth had been 
enrolled, a number of employers had been reached, and job sites were 
being developed. The funding came through, and the KD aspects of the 
project went almost immediately to that region to which things go in 
handbaskets. 

The youth staffer who had written the plan resigned withouT notice. 
The CETA director deliberated, decided to go on with the program, and 
hired a new youth director. First day on the job, this latter official 
reported to the prime's director that he . • could not find the 
EtyjC Knowledge Development plan." J^7 
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Nor, from that point onward> could any other observer. The 
program went forward, and in rather good fashion: 40 potential 
dropouts were tanrolled in the in-school track, and 90 dropouts in 
the out-of -school component. Eventually, 83 of the youths either 
graduated or obtained their GED, and 30 young men and women were placea 
in private sector jobs. 

But the KD elements, written so carefully into the original project 
plan, were not seen again. The entire control group mechanism was never 
instituted, and pre- and post- testing and other measurement devices 
ver>2 also abandoned. 

The project, then, was a success operationally — young people 

were helped — but a bust insofar as effective KD is concerned. In an 

after-the-fact acterapt to gain some sort of lemming from the project, 

the prime commissioned an .valuation by an outside cbserver. That 

observer recalls: 

They called me in — in July or August — 
and told me: 'Write something.* It was 
like that. 

His work did manage to salvage some learnings from the project — good 
.reportage on the achievement of interinstitutional linkages, the 
administrative difficulties such linkages can engender, and so on. There 
was not, however, a great deal of hard, KD-type learning to salvage 
because the impetus which would have produced such learning was swallowed 
up early on by overriding operational considerations. 

The implications of this sort of conceptual failure will be touched 
on in more depth later in this report. Still, we must at least mention 
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fit this point our conviction that a strofig infusion of technical assistance 
could have saved the Knowledge Development aspects of the Payette County 
youth, demonstration* 

The Jackson-Josephine (Oregon) project affords yet another example 
of the dangers Inherent in the comparison of small groups r In this case, 
ve can see how participant attrition can expunge almost the last vestiges 
of usefulness from such comparisons. 

Jackson-Josephine is in the enviable position of having a rather 
bullish labor market which alio wis the prime to place youths in job*^ 
£?"'rly easily. Because of this, the prime became interested in trying 
some new approaches and measuring them against traditional youth-job 
matching techniques. That is, Jackson-Josephine moved itself in the 
direction of Knowledge Development, whether or not it recognized the 
tcnrie 

The vehicle chosen by the sponsor was a vocational exploration 
program (VEP) for youths aged 16 to 19. The research aspects of the 
project cent ere j on dividing participants into two groups and comparing 
progress and results. The youths in the first group were to be placed 
ill private sector jobs; those in the second were to be assigned to a 
YCCIP unit. 

This design struck MDC as a sensible one — one from which useful 
learnings about the effects of the two markedly different work 
environments could be extrapolated. The implementation of the design, 
however » In large part vitiated its potential usefulness. 

The problem was, simply, one of scale. MDC learned that each 
of the two groups in the VEP experiment consisted initially of only 
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twenty youths. Any generalizations drawn from a two-group project with 
a membership of only 40 are perhaps automatically suspect. Unfortunately, 
they become r»ven more suspect when membership dwindles to 12 in each group, 
as occurred in the Oregon project. 

Here again, then, MDC has seen a commendable effort at Knowledge 
Development activity rjarred because of a fundamental mismatch between 
a research ctructure and the scope of data fed into the structure. 



MDC has dwelt at some length on incidents of largely unsuccessful 
Knowledge Development attempts involving highly structured comparisons 
of relatively small groups because such incidents appear to represent an 
all-too-frequently pursued dead-end in the first two years of youth 
program Knowledge Development. A large number of prime sponsors — possibly 
trying to follow what they took to be a suggestion from Washington — applied 
the comparison technique to two small groups, one consisting of CETA- 
cligibles and the other of a mix of CETA-eligibles ^d higher income 
participants. 

The issue of whether programs work better with CETA-eligibles only 
or with eligibles "mainstreamed** with non-eligibles is an appropriate 
question for national evaluation. The likelihood of determining so 
difficult an issue, through local studies of small groups of youths, however, 
is not high. Even under the best of conditions, results are likely to 
be ambiguous. The exercise simply has a low probability of providing 
useful information for local program purposes, and that probability 
dwindles as a predictable percentage of participants drop out of the groups 
being compared. 
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Very Small Mills, Indeed 

The second essential group of problems MDC believes it has 
identified among primes attempting KD is associated with the use of little 
or no structure for evaluation. In these cases, a certain amount of 
information was gathered, much of it potentially useful; but ultimate 
learnings were severely limited by the lack of forethought exercised. 

An example of this mode is Dennison, Texas, where MDC was referred 
to a high school potential dropout program operated through the local 
Council of Government. 

There, the subjects were a group of youths who had, except for an 
occasional visit to the high school, already dropped out. The prime 
sponsor, working through regular school personnel, organized a special 
class for these young people. The class spent half of each day in the 
school, working on academic and motivational subjects with a special 
instructor/counselor. During the second half of the day, the class 
functioned as a work crew performing park clean-up and similar teskiii. 
Members were paid the minimum wage for their work. 

After talking with the instructor and a number of the youths 
involved in thi» program, MDC's observer Cv?ncluded that the effort was 
first-rate, that the instructor had established rapport with the youths, 
and that the project had managed to turn a number of potential and 
actual dropouts into achievers. 

But what knowledge was generated from this program? Is it only 
that the richt kind of instructor (or supervisor) can mocivate youth to 
succeed where others would fail? And that the "right*' kind of instructor 
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was one who showed that he cared what happened to them? ("How come 
you care so much about how 1 do in school," one of the youths asked 
this instructor, "when my daddy doesn't?") 

These are not idle learnings, but yet rudimentary ones, of the 
sort that program operators have been assimilating over the years 
without benefit of statistical proof. Knowledge Development in such 
a program ought ideally to try to isolate factors that make the 
program succeed—the relative value of tLe screening process for 
program participants, curriculum, the work experience aspect, etc. 

At the least, a participant questionnaire or— better yet— a 
thoughtful, detailed exit interview followed by a compilation and 
analysis of results could have proved extremely useful in such a pro&ram. 
A small program is at a disadvantage when it attempts to project 
generalizable learnings of a statistical nature, as we have observed; 
but it has a distinct advantage over larger programs in that the entire 
program group can participate in interviewing and can be examined 
through follow-up for program impact. 

Too many prime sponsors seem to have ignored the opportunity of 
learning directly from participants, possibly because they thought of 
this kind of information as being "soft." 

MDC found another example in which a KD effort was hampered by the 
lack of formal structure in a program of the Escambia County (Fla.) 
prime sponsor. The program involved the provision of employment-related 
services to approximately 30 mentally retarded but educable CETA-eligible 
young people. 22 
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At the outset, the project included no identifiable mechanisms 
for information gathering andi no staff for performance^ aosessraent. This 
may have resulted at leawt partly from the hurry-up start the project 
had, since it was funded in part by an allocation directly from the 
office of the Governort 

Whatever the reason, once the project was under way, staff realized 
its informational shortcomings. To help correct them, an attempt was 
made to collect impressions of the program from participants and project 
supervisors. The avow(»d goal of this attempt was to produce feedback 
which would be u«7,eful in future planning. 

The attempt, however, was carried out only sporadically, and 
without the firm underpinning a fairly formal questionnaire survey 
methodology would have furnished. Thus, while staff did form some 
impressionistic opinions about possible project improvements, the lack 
of a structured approach to data acquisition and analysis kept this 
KD effort from being more than marginally useful^ 

As a final example of knowledge lost because of loose structure, 
MDC would cite briefly the multi-purpose youth employment center 
operated by the Montgomery (Ala.) prime sponsor. 

The basic thrust of the Youth Center is not job placement as such, 
but rather employability assessment and pre-employment assistance. 
Each young person who eniers the Center goes through a rigorous 
assessment and testing program, the results of which guide the 
formulation of individual career development plans. The career 
development plan details the specific sorts of assistance the individual 
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youth needs to become job-ready: additional literacy training^ 
counseling, work experience, or whatever* 

The Center is an undertaking of some magnitude. At the time of 
MDC's site visit, approximately 700 young people had been referred to 
it by ESC and the prime's own intake office. Yet 'he Center has 
produced practically no documentation; there has apparently been no 
systematic assessment of the various services provided by the center, 
and statistics in even such basic areas as different types of participant 
terminations are sketchy. 

It is not that the Center's staff do not recognize the need for 
a more structured approach to Knowledge Development activity. Rather, 
it is a matter of operational consideratlonfi overriding learning 
opportunities. There is also-the element rf need for help in designing 
and implementing a learning framework for the project. The Center's 
planner, in fact, voiced a direct request for technical assistance to 
MDC's observer during the latter 's site visit. 

Some Indications of a Better Natch Between Mill and Grist 
The Clark County (Wash. ) prime sponsor is one of several which has 

done a pretty good job of matching up its analytical framework and 

informational input with respect to KD activities— -this despite the 

aforerin'tioned time bind in which it operated. 

Initially, the prime planned two KD efforts — one as a component of 

its YETP in-school program, and one as part of its out-of --school youth 

project. 
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Because of operational dif ficr.lties , tha in-school segment never 
flew. The one-month lead time simply did not enable the planners to 
impose a workable administrative structure for a program involving 9 
schools. Moreover, due to differences in the funding year and school 
year calendars, project staff had to try to implement a new program in 
the midst of an ongoing semester, and th^i resultant scheduling difficulties 
were never overcome. 

The out -of -school effort went more smooth?. . This project centered 

on the school-to-work transition and set out to assess whether youth 

"J 

would be helped more by a combination of transition services.r(classroom 
and OJT trainings career orientation, and work experience) than by a 
single service (traditionally, work experience alone). According to 
project staff, a number of measurement techniques were written into the 
program: pre- and post-project occupational awareness tests, unsubsidized 
job placesnent rates, entry wage levels, and participant evaluation 
through interviews. 

Much of this measurement grid was never instituted, whatever the 
reason. Outcome assessment was carried out, but through much more informal 
means than the plan had set forth. Project staff ended up monitoring 
placement rates, wage levels, nature of employment, and job attendance rates 
for program graduates. 

Even this reduced assessment mechanism, however, led to an important 
and program-related learning: the young people who received the new, 
comprehensive transition services package showed a higher positive 
termination rate, a higher job placement rate, and higher starting wages 
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than had the youths who had participated previously In the single 

service, work experience model. 

That, It seems to MDC's observers, is a pretty sound piece of 
Knowledge Development, even though the technical aspects of its 
production were less than academically pure. 

MDC would also list Yakima. Wash., among the prime sponsors who 
seem to be doing a fair job of matching up research structure and 
information handling in their KD effort. 

Perhaps the best procedural lesson to be learned from Yakima's 
experience is that the prime there selected a simple KD-oriented 
question to try to answer and adopted a straightforward method for 
seeking that answer. 

Schematically, here is the picture: 

—The I. rime posited an item of "received wisdom": 

A major factor in the unemployment of youth is their lack 
of hope at being able to get a job. 
— The prime asked a question: 

Is that statement valid? 
—The prime subcontracted for a survey by questionnaire to 
answer its question. 
The questionnaire was administered to every young person who applied 
for enrollment in the local YETP over a number of months. Approximately 
325 questionnaires were completed. The document included some adjunct itens, 
but the key question was this: 

What do you feel about your chances of entering and staying 
^ In the business world, and why? 
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Interestingly, less than 3 per cent of the respo-adents answered 
that they considered their chances •'poor.'* Fifty-one percent felt 
that their chances were "average," and A6 per cent characterized their 
chances as "good." 

The very simplicity of this KD exercise might lead some observers 
to write it off. They might argue: Surely, anything so unsophisticated 
cannot really be valid Knowledge DcvelopmeTit . hT)C would not agree with 
that position. Admittedly, the procedure wf»s simple. Still, it 
bears the hallmarks of true KD: It produced and documented new 
information, and the information has relevance for future program 
development. [It would be less than smart, for example, in view of ^he 
questionnaire responses, for Yakima to include a major "overcoming 
negative self-image and feelings of incompetence" component in its 
future youth employment activities. ) 

An interesting and slightly diffarent application cf numerical data 
as a tool for Knowledge Development can be eeexi in oparfition In the 
youth program of Richmond, Calif. Thd difference is piCbably directly 
attributable to the project director's somewhat idiosyncratic conceptualization 
of Knowledge Development, which she views in the context of " . . . something 
competitive, [a way of] seeing whether certain services to the target 
group make a difference." 

The Richmond Youth Employment and Training Program has an enrollment 
of approximately 150 youths. All of the clients have either dropped out 
of school or are potential dropouts; over half are offenders, and there 
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are also sizable components of unwed mothers and handicapped youth. Th? 
client group is spread among all 7 of Richmond's high schools, but the 
largest contingent~65, or A3 per cent— attends the special "continuation" 
high school, where the entire student body consists of "troubled'* youth* 

The initial philosophical stance of the^project's staff was, 
unabashedly, that of service providers. They saw themselves as helpers 
of seriously disadvantaged youth through the provision of work experience 
opportunities and counseling and placement services. 

Once the program was under way, however, the director began 
making some comparisons among program clients and non-clients. She 
began, in her opinion, to look at Knowledge Development matters. The 
basic matrix of comparison was the performance of the YETP enrollees 
at the continuation high school as opposed to the performance of the 
rest of the student body. 

The comparison showed that, for the school year ending in 1978, 
only 34 per cent of the non-client population ©ither graduated or 
remained in school with passing grades. On the other hand, every one 
of thr 65 YETP enrollees either graduated or returned to school the 
following year, and all of the latter group maintained passing grades. 
And there were good specifics within the overall good numbers set: 

~0f the 8 unwed mothers enrolled, A graduated and A returned 

to school the next year. 
—Of the 9 enrollees with records, none was involved in 

serious problems with the law while in the program. 
—Eleven handicapped youths from the program were helped to get 
and hold wage-paying jobs for the first time in their lives. 
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The Richmond effort vas, then, a demonstrably effective program; 
Its KD elements, though interesting, were quite rudimentary. This 
appears to be the case because of two factors which MDC also saw in 
operation in many other site studies: The KD effort was tacked on, and 
it was stopped short. 

The program was undertaken with no integral resv^arch design built 
In. The basic numerical analyses that were performed were added later, 
when the obviously competent staff stood away from service delivery long 
enough to realize that some sort of outcome assessment was necessary. It 
Is thus not surprising that the production and use of statistical 
Information was very simple. 

Relatedly, it Is not surprising that the statistical analysis 
stopped at a very early poin'Cv The numbers showed that the youth 
In the special program performed better than non-program youth, and that 
. was left to be that. Perhaps real Knowledge Development would have 

come only with the next logical step in analysis (a step not yet taken by 

Richmond): Wliat was there in the treatment given the project youth 

that caused or helped them to perform better than the other young people? 

This, of course, is the kind of question all KD efforts must 
address if progress is to be made. Once again, MDC would point out its 
strong belief that Richmond would be a good deal farther along that 
particular KD trail if MDC — or somebody — had been able to help them, 
rather than Just interviewing them. 

A good example of how fairly basic interviewing and statistical 
techniques can be used to provide a somewhat mora sophisticated level 
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of Knowledge Development than Richmond's can be found in the San Antonio, 
Texas > programs operated by the Bexar County Labor Community Agency 
(an arm of the city's AFL^CIO Council). 

BCLCA's programs feature the programmatic interaction of handicapped 
with non-handicapped youth and are structured so as to produce learnings 
about this admixture. In its Fiscal 1978 Summer Youth Employment Program, 
the organization utilized pre- and post-testing and interviewing 
techniques — first to compare attitudes about the world of work and the 
handicapped and to develop self-image profiles; and second to get 
in-depth participant and counselor evaluations of the program itself. 
A number of program changes were made as a result. "Among other things," 
the program director told us, "we found that pairing a handicapped with 
a non-handicapped youth didn't always work. We were a little disappointed 
with thatf but we also realized that we had done it somewhat hastily." 

The Fiscal 1979 SYEP involved 170 participants, of whom 103 
were classified as minimally handicapped and 67 as non-handicapped. All 
participants received 24 hours of orientation, three hours a week of 
counseling, and a total of 240 hours of work e:;perience. 

This time program staff took care with the pairing of the handicapped 
and non-handicapped youth, not doing it as a matter of course, but only 
when It seemed suitable both from the standpoint of the work and of the 
youths involved. Both counselors and participants expressed heightened 
satisfaction with the technique in post-program interviews. Where only 
67 per cent had found it satisfactory in 1978) 80 per cent found it 
satisfactory the following summer, and these results reflected the 
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perceptions of top project staff. A probable ancillary benefit of the 
Interviewing process was the increased awareness on the part of 
counselors of the ability of handicapped youth to hold full-time jobs 
(91 per cent compared to Fiscal 1978 's 67 per cent). 

The learning here is but one of a number gained from careful, 
structured use of questionnaire and interviewing techniques — others 
included the knowledge that the orientation session was boring and 
unnecessary to a number of the participants and needed either to be 
re-structured or made optional; and that the three-hour weekly counseling 
sessions were too long for most participants and should be reduced — at 
their suggestion — to two hours. 

In its YETP program for Fiscal 1979, BCLCA mixed 42 minimally 
handicapped graduating seniors with 62 non-handicapped graduating 
seniors in a program consisting of orientation, skills training, and 
jobs. The knowledge development function was structured to test the 
worth of an Adult Performance Level component (basic life skills — i.e. 
government, economics, child care). Half of a representative random 
mix of handicapped and non-handicapped youth were assigned to the APL 
and the other half to a consumer education activity. It was also 
structured to compare in 30-60-90 day followups the placement and 
sticking rate of the handicapped to the non-handicapped participants. 

Results seemed to indicate that the APL component is worthwhile. 
At 30-day followup, 82 per cent of the APL non-handicapped group were 
on the job, compared to 33 per cent of the non-APL group; 65 per cent cf 
the APL handicapped group were positive, compared to 27 per cent of the 
non-APL handicapped group. 
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The comparison of positive termiQa^^ions between the handicapped 
and non-handicapped groups la more difficult to weigh. At the end of 
the 30-day followup, non-handicapped positive terminations were at 90 
per cent) while handicapped positive terminations were at 61 per cent. 
Is this a good positive termination level for handicapped? Program 
staff are not sure — "in fact, we want to see the results of the longer 
term followups before we get very excited about even the non-handicapped 
rate," one said. At any rate, a handicapped positive termination 
rate at least has been established for future comparisons with other 
similar programs and with programs for the handicapped only. 

In effect, then, Bexar County has taken some fairly basic tools 
and produced good early learnings that have, in turn, been plugged 
immediately back into the prpgrara design and operation process. The 
sophistication of the devices employed may well have been limlteii; but the 
results generated and the appllc^^tion of those recults are, most emphatically > 
embraceable under the rubric of Knowledge Development. 

And Some Further Indications with a Firm Maybe 
Here and there, MDC has found some other second-year KD activities 
which have not yet yielded solid results, but which may prove instructive 
In the future. Brief treatments of several of these follow. 

The St. Louis prime sponsor set itself a rather impressive youth 
program Knowledge Development docket for 1979. The work was to be 
Carried out by St. Louis University's Center for Urban Programs and was 
to entail the monitoring of implementation issues and the provision of 
Impact evaluation for 5 YETP programs mounted by the prime. 
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Among the KD-related matters to be addressed were the following: 
--Organization of YETP projects. How are the projects 

organized? What differences inhere in the 12-month approach 
fis opposed to the school-year approach? 
--Selection of participants. What groups of youth are 

served? How are they recruited and selected? 
— Provision of services. What mechanisms are used to 
select services matched to desired program outcomes? 
—Private sector participation. What techniques were used to 
encourage private sector involvement? What sort of work 
sites and experiences resulted? 
—Coordination. How were YEDPA activities meshed with other 

existing -youth programs? 

The basic methods of addressing these questions will be ongoing site 
visits and interviews with project pen'onneA 

Because of scheduling problems, the sponsor and the subcontractor 
both realized that the learnings from this KD effort would be of only 
minimal use in the FY 1980 program design process. However, all the 
principals feel that the knowledge developed will be applied extensively 
to program planning, designing, and funding decisions for FY 1981. 

The San Francisco prime sponsor, according to its director of 
youth programs. Is constantly engaged in some form of KD activity, and 
has been for a number of >ears. Of course, that statement reveals 
something of that particular official's construction of the tfiTn "Knowledge 
Development,'' which he expounded on to MDC's observer: 
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KD isn't really all that new a thing. Back in the 
early '60's, the Labor Department and the Ford 
Foundation funded an experimental Youth Opportunities 
Center, which grouped the activities of about 10 
agencies. They called it a 'Knowledge Development 
Pilot Project.' 

His further conversation showed that he held KD to be synonymous with 
any innovative program approach, and with research in general. It is 
perhaps also significant that, in his memory, a major outcome of the 
"KD Pilot Project" had been the conviction that it is difficult to 
combine an action program with a research project, "... since the 
latter keeps getting in the way of the former." 

The most interesting locus of San Francisco's KD effort^ has 
probably been the twenty per cent set-aside "special projects", which 
the prime has carried on for three summers now. At the sponsor's own 
admission, it has not attempted to do any "sophisticated research" in 
these projects. But the projects do qualify as KD efforts, under the 
prime's view of the concept, since their basic tenet has been to try 
out innovative approaches to youth programming. In several cases, 
these new summer techniques have seemed successful enough so that they 
have been added on to regular, school-year CETA youth programming. 

Perhaps the three major contributions of the KD-oriented special 
projects have been these: 

— The prime has learned that innovative summer programs can 
attract and enroll youth who were not drawn to the ongoing, 
regularized school-year programs # 
—The prime has learned that a program featuring a mix of 
services (job readiness, literacy upgrading, career 
^ orientation) can serve certain youths better than a one-track 
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—The prime has learned to Identify CBOs which have the 
expertise necessary for providing services to the most 
severely disadvantaged and handicapped youth. 
Finally, It is Interesting to note that the prime and the regional 
DOL office have been at loggerheads over the special projects since 
their Inception. Prime staff state that regional officials have never 
been comfortable with the Innovative features of the special proj'^cts, 
fearing that these features, in several ways, bend the regulations 
regarding eligibility and allowable activities. 

Because of this, the San Francisco staff feels federal staff should 
stay out of the Knowledge Development arena, lotting the local primes go 
it alone. "If the Feds had it their way," said a staffer, "the special 
projects program would have been disal lowed > and our Knowledge Development 
initiative would have b-aen reduced accordingly." 

The Atlanta prime sponsor cites as a major KD effort a progression 
of events which adds yet another nuance to the overall story of the Kr 
concept: Knowledge Development as a firefighting tool. 

Atlanta's 1978 summer youth program was in deep trouble. Those 
were the hectic days of "fraud" and "abuse" headlines, and the Atlanta 
program — whether or not deservedly — came in for more than its share of thera. 

The prime took a courageous path in the face of these problems — a 
path It feels led to considerable Knowledge Development. It called in an 
outside evaluator to look at its entire youth programming effort. 

The evaluator evaluated and, by fall, submitted a detailed final 
report containing 38 specific recommendations to the primers Planning 
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Council. The Coimcil assigned the report to a subconmittee for review 
and assessment* and the press and some segments of the public smirked 
knowingly, "Well, that's that." 
But it was not. 

The subcommittee worked over the report In a series of meetings 
lasting from October of 3 978 through May of 1979. In addition to the 
evaluator's report, It reviewed staff papers, citizen complaints, and 
its own observation about the previous summer's youth program. 

In early summer of 1979, the subcommittee put forth and the full 
Planning Council endorsed a slate of recommendations which called for a 
laajor overhaul of the summer youth program. The following exceppts from 
the subcommittee's report indicate the broad scope of the slate of 
recommendations! 

--Change to a centralized administration and a centralized 
payroll. 

-*-Make provisions for administrative assessment and process 
analysis of the program and individual projects throughout 
the summer. (During the first weeks of the 1979 summer 
program, staff spent the first hour of each day in process 
evaluation. ) 

—Develop new procedures and instruments for work site 

monitoring. (In 1979, staff for the first time did written 
evaluations of work sites and scheduled regular meetings 
with work «tte supervisors.) 
~Change to a centralized recruitment and Intake certification 
and referral procedure to increase, participation among 
ERJC target groups. 



36 



- 35 



The Planning Council's recotcmendations were not merely acknowledged 
by the prime, but acted upon. The '79 summer youth program was 
redesigned, and it operated in a relatively trouble-free manner. 
Knowledge ; Development was hailed as a saver of bacon. MDC hopes that 
the prime will go on from there to the realization that KD, once 
Institutionalized, can also serve to prevent situations in which 
bacon-saving becomes necess^ary. ' 

* * * * 

With the site visits finished, we had to ask ourselves whether 
there was reason to change our original view of what Knowledge Development 
o tp^ht to be— the collection of information In some planned fashion to 
develop learnings useful in program design or operation. 

We could see no reaf Dn to alter that definition, but we had reason 
to examine more closely with the benefit of experience what form that 
knowledge might take. 

The form can be rudimentary, as in the case of the Yakima learning 
that the youth they are serving do not have built-in expectations of 
failure. This may not seem like a great learning but it did come as 
a surprise to the prime sponsor. 

It can be somewhat more developed, as in the case of Richmond 
County where statistics reinforce the value of the program but do not 
provide much insight into the why's and wherefores. It would be a 
mistake to underestimate the morale value of statistical support for the 
value of the program and encouragement to make improvements in the 
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program. "It's great to have real evidence that you're getting your 
act together," as one prime sponsor director said. 

It can be rather more substantive without bein,;: fully developed. 
The San Antonio SYEP Knowledge Development function already has yielded 
valuable information which has been put to good use in program re-design. 
Further efforts at pre- and post-testing and interviewing may bring 
further progress. Further follow-up on the YETP program described here 
could lead to a program mixing handicapped and non-handicapped youths 
to the optimal advantage of both groups. 

It should be clear that MDC found no CETA prime sponsors with what 
could be described as fully articulated Knowledge Development plans. 
That Is hardly surprising, given the fact that the function oE Knowledge 
Development in youth programs is only two years old. 

It may be more useful to observe — because it is also true — that 
we found a number of prime sponsors at various stages of experimentation 
with Knowledge Development approaches. And we found others who were 
already beginning to log knowledge, who had proceeded far enough down 
the line to begin to learn more about how well their programs work and 
how to improve them. 

The big question remaining is how to build on the beginnings. With 
approximately 460 CETA prime sponsors in the field, how can Knowledge 
Development be stimulated? How can it be seeded where it does not now 
exist? Holt; can those who have begun get assistance in continuing? 
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IV. .SuTmnary:__and Rgcoigmendations 

As a result of the work done under this contract, including 
contact with regional offices and brief ejq)osure to prime sponsors who 
are not engaged in Knowledge Development, as well as with those whose 
efforts are reported here, It is possible to make several general 
observations : 

— There is no comnon definition of Knowledge Development 
anywhere-- no general agreement at all qn what it is or 
how it i^ to be achieved. 

— There is limited activity under the rubric of Knowledge 
Development in youth programs operated, by prime sponsors 
around the country— more in Fiscal 1979 than in Fiscal 1978, 
but not much by. any standard of measurement. 

— If the OYP's goal for local Knowledge Development is to 
create ferment at the prime sponsor level — casting about 
for lessons worth learning even at the cost of initial 
confusion — progress can be reported. If the goal, howaver, 
Is to spur prime sponsors to evaluation of youth programs 
for management decision-making purposes, much work remains 
to be done. 

— Despite limited progress, the average prime sponsor is 
Interested in learning more about its youth programs so 
that they can be improved, and would welcome advice, examples, 
and technical assistance in any form in which it might be 
offered. 
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This last point deserves underscoring. MDC staff and field 
associates found everywhere a willingness on the part of prime sponsors 
to get into Knowledge Development~as they und-^rstood it. In the case 
of those who had not accomplished much, there was a sense of frustration 
and often a tendency to blame regional and national officials for not 
offering more definition and help. 

Clark County's unhappiness with the time frame in which its KD 
effort was squeezed did not prevent it from getting something under way, 
but a lack of technical assistance probably foredoomed that effort. 
Jackson-Josephine's federal representative undoubtedly was being honest 
when he told the prime sponsor that he didn't know what Knowledge Development 
was, but it is hardly likely that the implications of his answer spurred 
the prime sponsor on to greater efforts. 

Even those who had literailly nothing in the way of a KD effort 
often had been gathering for some time the kind of information that could 
be used for one. The prime sponsor in Texas with all the tests and 
the willingness to "develop all the knowledge you want" was in need, 
first, of a definition, and then, of some rudimentary assistance to get 
something started. 

Where KD of a limited sort had been gained, it was often relatively 
easy to see how more of a useful sort could have been gained with little 
extra effort. The learning in Yakima that program participants did not 
have negative expectations in the job market was fine as far as it went, 
but a good deal more of value could have been learned in the same survey 
sampling* What Yakima lacked was a well-defined research plan designed 
to get the most out of the ef fort***-'more knowledge for the same bucks, 
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These are all instances where a little sound advice » a few hours 
or days of counseling would have been enonsously h^^lpful. Not 
surprisingly, as we get to the primes where a bit more was learned, a more 
considerable effort may be required to provide commensurate help. 

An example here is Richmond, Calif., where the operator proved rather 
convincingly that her program was helping dropout youth, but had built in 
no research design Intended to tell her why. Creating such a design is 
no simple matter— learning "why" Is almost always more difficult than 
learning "what" — but it is not beyond the reach of even the smallest 
prime sponsor working with others experienced in local program evaluation. 

Working with certain more advanced priciia sponsors affords the additional 
bonus of an opportunity to develop model systems of Knowledge Development. 
The San Antonio prime sponsor •s efforts to learn how best to help 
handicapped and non-handicapped youth in mainstreamed programs is one 
that could conceivably have national importance. What San Antonio needs 
Is a little encouragement, reasonably apt advice, and modest technical 
assistance. "Nothing big," as the subcontract director told MDC, 
"but a pat on the back and a helping hand weM never turn down." 

In addition to turning up a willingness of prime sponsors to work 
on Knowledge Development, this study provided an outline of the kind of 
assistance that is most needed in a variety of forms suitable for a 
variety of settings. 

First, some firm definition of Knowledge Development as applicable 
to youth programs at the state and local level should be provided » We 
have no pride of authorship in our working definition: the collection 
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of information under a plan intended to yield knowledge that may 
predictably be useful in improving CETA programs for youth. Absent 
a better or ©ore authoritative definition, we offer it for purposes of 
discussion. 

Once KD is defined, prime sponsors need, help in designing evaluation 
plans that are within the realm of accomplishment and that may reasonably 
be expected to provide knowledge useful to them. We would stress both 
. the utility of the knowledge and the phrase -^V^tp them" with the conviction 
that only at the local level can decisions be made that effect program 
improvement. 

Too often, MDC observed KD initiatives designed as though the 
knowledge to be gained would be useful to someone at the national level — or 
at least someone remote from the program scene. And too often, again, 
we saw KD initiatives that were either too fuzzily conceived to produce 
useful results or too ambitious to be carried out realistically. 

The first rule of evaluation is to set goals that can be evaluated. 
Under these goals, quantifiable, measurable objectives must be ordered. 
In adapting this procedure to KD, the prime sponsor should ask itself: 
What are the specific learnings I intend to gain in this effort? Is 
the knowledge I am likely to get worth the effort I expect to have to make? 

It seems clear to us that many if not most prime sponsors will need 
help in making the wisest possible decisions here. Whatever technical 
assistance is offered will have to take into account the teaching needs 
suggested here--and also the fact that the best teachers of program 
operators probably are other program operators. 
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To attempt a technical assistance effort intended to help everyone 

in the field would be extrenely costly even assuming the teaching 

resources were available. A much more modest effort would seem Zo 

suggest Itself: 

~A certain amount of information should be circulated 

routinely through the regional offices to the prime 

« 

sponsors in order to clarify how local KD is perceived 
nationally. 

•-•-Modest amounts of technical assistance should be provided 
to prime sponsors directly. This T/A should be granted 
on the basis of the willingness of the primes to set 
worthwhile and achievable KD goals and to put their own 
resources into the effort. It should be flexible, so as 
to permit the agency providing technical assistance 
freedom to put its T/A where it is likely to do the most 

. good. It should be sufficient in scope to have a 
"seeding" impact nationally, 

•-A communications network should be erected so that prime 
sponsors can be helped initially to share their learnings 
in KD. Eventually, such a network ought to be built Into 
the natlonal-reglonal-local CETA structure for a variety 
of informational purposes • 

--Regional workshops should be organized around the better 

KD efforts emerging as a result of the technical assistance. 

Here, relatively successful prime sponsors would provide a 
learning resource for other primes in their geographical 
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area, with the agency providing the T/A supervising and 
collating information. Care should be taken to involve the 
regional offices in thU experience. 
—Consideration ohou2d be given then to the convening of a 
national conference whose goals would be the further spread 
of major learnings and decisions about next steps. 
It should not escape notice that the Knowledge Development effort 
for youth programs has critical implications for other CETA and non-CETA 
employment and training initiatives. However it is finally defined, KD 
seems clearly to be another way of describing local evaluation. 

If it can be said— as we think it can— that the state of the art 
of KD reflects only scattered, unformed effort, not much more can be 
said of the average CETA prime sponsor's local evaluation effort, period. 
If KD were given a boost, it would almost certainly provide learnings 
useful in programs other than those designed exclusively for youth. 

Even more important, the structure ultimately necessary to stimulate 
KD learnings, to spread the most useful of them— a process that could be 
called "knowledge sharing"— can be designed, and indeed should be 
designed, to support all of the knowledge that CETA program operators 
can develop. 
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